Decison Boundary and hyperbolic unfolding¶

Decision boundary of a neural network¶

Dataset with two classes

Train a neural network on the binary classification task

In [8]:
# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data, label, n_epochs=15)
epoch train_loss valid_loss accuracy time
0 0.725640 0.717628 0.491000 00:00
1 0.676842 0.639662 0.491000 00:00
2 0.560278 0.509667 0.909000 00:00
3 0.466550 0.434938 0.926000 00:00
4 0.404259 0.381256 0.960000 00:00
5 0.353271 0.338769 1.000000 00:00
6 0.329343 0.324409 1.000000 00:00
7 0.320989 0.319070 1.000000 00:00
8 0.317565 0.316634 1.000000 00:00
9 0.315937 0.315435 1.000000 00:00
10 0.315094 0.314812 1.000000 00:00
11 0.314643 0.314483 1.000000 00:00
12 0.314412 0.314322 1.000000 00:00
13 0.314310 0.314262 1.000000 00:00
14 0.314281 0.314253 1.000000 00:00

Contour plot of the neural network function

The descion boundray is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary¶

First Approach: Sampling algorithm¶

  • Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
  • Implementation by Matteo.

Gradient flow method¶

  • Pushing point to the decision boundary.
  • Using automated differentiation for computation graphs of tensors.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding¶

Entanglement problem for tori¶

In [ ]: